A variety of techniques are used to reduce write latency, which is the amount of time that is required to write data to memory of a computer system. When data is written to memory, the data typically includes metadata and actual data. The metadata describes attributes of the actual data. During a typical new file data write operation, the metadata is conditioned (i.e., generated/written, modified or deleted) by the file system (FS) running on the computer system as the actual data is being written to memory. This conditioning of the metadata requires accesses to memory during the data write operations, which adds latency to the write process that is detrimental to the write performance of the computer system.
One known solution for mimimizing FS latency is to implement what is generically known as a custom FS of a type designed to minimize the need to condition metadata as the computer system is writing data to memory. This solution has many disadvantages, including, for example, achieving a correct design and reliable implementation of the custom FS, the need to learn the intricacies and limitations of the storage hardware, the need for extensive verification testing, and the need for the user to learn yet another file system and its controls.
Another known, but less well understood solution (and normally unused due to its use model deficiencies) is known as a “pre-write/write” approach. With the pre-write/write approach, a file of the size to be captured is pre-written with “dummy” data to pre-condition (i.e., write out) the metadata describing the actual data and then the file pointer is re-wound to the beginning address at which writing of the dummy data began. The actual data is then written over the dummy data. If the actual data to be captured is of a very short duration and of known size, and there is not a need for the immediate start of writing, then the pre-write/write approach may be acceptable from a user perspective. Of course for file writes of, for example, an hour in duration, then the initial pre-write pre-conditioning pass may take an hour or longer, which would not be acceptable for nearly all realistic customer applications.
The pre-write/write approach is implemented in a storage performance benchmark tool called CrystalDiskMark that may be used to test the performance of the underlying storage hardware while minimizing the performance degradation caused by the FS. Users of the tool generally understand the length of time they are required to wait to receive results, and the test runs default to relatively short time durations. For each file write test performed by the tool, a “Preparing” status indicator is displayed to the user during the pre-write stage and a “Write” status indicator is displayed to the user during the write stage. These indicators are displayed for roughly the same amount of time, and therefore the user is made aware of the amount of time that is required for the tool to precondition the metadata.
In signal measurement systems that are used in high end, high bandwidth applications, such as, for example, test and measurement equipment and radio telescopes, the FS must be capable of performing write operations at very high speed. This is especially important in gapless, or latency consistent, applications, such as streaming applications. Through experimentation, it has been determined that the storage hardware in such systems may be fast enough to meet these demands, assuming that state-of-the-art hardware is used, but that standard, or non-custom, FSs (e.g., Windows NTFS) are not fast enough (impose too large of an additional performance overhead on the hardware in updating metadata) to prevent the hardware pipeline from overflowing with reasonably cost-effective hardware components and designs.
Accordingly, a need exists for a computer system that runs a non-custom, or standard, FS that reduces write latency in computer systems used in high end, high bandwidth measurement systems. A need also exists for a computer system that runs a non-custom, or standard, FS and that is particularly well suited for reducing write latency in gapless, or highly latency consistent, applications, such as streaming applications.